1. Field of the Invention
The present invention relates, in general, to an error concealment method and, more particularly, to a hybrid error concealment method, which adaptively combines a temporal error concealment algorithm, applied to intra-frames, with a spatial error concealment algorithm, used in the H.264 standard, so as to combine the advantages of spatial and temporal error concealment methods with each other, thus transmitting real-time video in a network environment in which the packet loss rate is high and the bit rate is relatively low.
2. Description of the Related Art
Generally, as transmission speed increases with the development of technology for links, such as Asymmetric Digital Subscriber Line (ADSL) or cables, the amount of video data transmitted over the Internet is gradually increasing due to a Video-On-Demand (VOD) service, a video streaming service, etc., and such video services have recently gradually extended to the Wireless Internet, ranging from digital broadcasting to mobile devices.
However, the Internet does not currently guarantee Quality-of-Service (QoS) for real-time video transmission, and has instability characteristics indicating that packets may be lost during transmission according to the characteristics of wireless networks, so that several technologies are required to code and transmit video data.
Moreover, video has requirements with respect to bandwidth, signal delay, and loss. First, suitable video quality can be maintained only when video has a minimum bandwidth. Second, in terms of signal delay, streaming video or real-time video must be continuously played without interruption, unlike typical data. Accordingly, if video data arrives late, rather than on time, loss may occur. Finally, if the loss rate increases to some degree due to bit errors or packet loss, video cannot be recognized, so that the loss rate should be considered during a procedure for encoding and decoding video.
In this case, the problem of bandwidth or signal delay is related to a hardware environment on a data transmission path. Accordingly, these requirements for bandwidth or signal delay can be easily satisfied by improving hardware, and can be predicted. In contrast, if loss occurs, the prediction of the occurrence of loss is impossible, and individual cases and extents to which loss occurs vary, so that separate measures should be provided in preparation for the cases and extents of loss.
Therefore, control methods for compensating for loss caused by the occurrence of errors, such as Forward Error Correction (FEC), retransmission, error resilience, or error concealment, capable of maximizing video quality even in the presence of packet loss, are required.
Among the above control methods, FEC is a scheme in which redundant information is added to a bit stream, and is disadvantageous in that, as the number of bits increases, channel capacity is decreased, and performance is degraded when packets are intensively lost for a long period of time. The retransmission method, such as an Automatic Repeat Request (ARQ), is problematic in that it is not suitable for real-time video transmission or streaming in terms of signal delay. Further, error resilience technology is a scheme of minimizing the influence of loss even in the occurrence of packet loss by considering packet loss in the stage of an encoder coding and compressing packets before the packets are transmitted. However, this scheme also has several problems.
Therefore, one technology that has recently received attention is error concealment technology, in which error concealment is performed by a decoder after receiving video data, and lost data is reconstructed using correctly received neighboring information so as to provide image quality suitable to be recognized by human eyes when packet loss occurs.
That is, a video stream is compressed (coded) to be easily stored and transmitted. Such a video stream coded in this way may frequently undergo data loss or may be corrupted during transmission due to channel errors and/or network congestion, and the loss/corruption itself of data appears as a lost pixel value at the time of decoding data.
In order to reduce such defects, a decoder compensates for the lost pixel value by predicting the lost pixel value from other macroblocks within the same image or other images. This operation is called “concealment” or “hiding”, and is advantageous in that it can reconstruct data relatively close to original data using the characteristic by which correlation with adjacent information is high, and it does not especially increase transmission capacity and thus reduce the burden of transmission when transmitting data to the network.
Therefore, if error concealment is applied, video quality can be recovered without increasing the burden of transmission costs in an environment in which packet loss occurs. Further, if the above-described error resilience technology is applied together with error concealment, more efficient error resilience can be implemented. On the basis of the above fact, it can be predicted that, if loss occurs in a network, the realization of video quality will be greatly influenced depending on the performance of error resilience and error concealment.
Further, since the preference of consumers recently tends towards compact systems, requirements, such as a very small size, real-time video processing, and excellent image quality, must be satisfied. Technology proposed to meet such requirements is encoding/decoding, that is, CODEC, technology based on the H.264/MPEG-4 Advanced Video Coding (AVC) (hereinafter referred to as H.264) standard.
Recently, a plurality of relatively new error concealment algorithms for improving on the performance of standard error concealment algorithms, which are adopted in H.264, has been proposed. For example, Jung et al. proposed an algorithm disclosed in [reference cited: B. Jung, B. Jeon, M. D. Kim, B. Suh, and S. I Choi, “Selective temporal error concealment algorithm for H.264/AVC,” Proc. IEEE International Conference on Multimedia and Expo, vol. 1, pp 411-414, June 2004] in which a lost macroblock is divided into the background and the foreground, whether temporal correlation is high is determined, and error concealment is performed using a plurality of reference frames in consideration of the fact that correlation is decreased when only a single previous frame is used, in the case of the foreground.
Further, Lee et al. proposed an algorithm disclosed in [reference cited: P. J. Lee, H. H. Chen, and L. G. Chen, “A new error concealment algorithm for H.264 Video Transmission”, Proc. IEEE Int. Sympo. Intelligent Multimedia, pp 619-622, October 2004] in which the size of a lost macroblock is determined, motion vectors are detected in a plurality of reference frames, and a motion vector having the minimum side match distortion is selected.
Further, Xu et al. proposed an algorithm disclosed in [reference cited: Y. Xu and Y. Zhou, “H.264 video communication based refined error concealment schemes”, IEEE Trans. Consumer Electronics, vol. 50, no. 4, pp 1135-1141, November 2004] in which, if the number of intra-coded macroblocks is greater than or equal to the number of inter-coded macroblocks, for adjacent macroblocks, a spatial error concealment algorithm is applied, whereas, if the number of inter-coded macroblocks is greater than the number of intra-coded macroblocks, a temporal error concealment algorithm is applied, so that an error concealment algorithm is adaptively applied depending on the features of adjacent macroblocks.
Further, Panos et al. proposed an algorithm disclosed in [reference cited: P. Nasiopoulos, L. Coria-Mendozal, H. Mansour, and A. Golikeri, “An improved error concealment algorithm for intra-frames in H.264/AVC”, Proc. IEEE Int. Symp. Circuits and Systems, vol. 1, pp 320-323, May 2005] in which intra-frames are independently coded and thus correlation with a previous frame may be decreased, but intra-frames are frequently coded to prevent the accumulation of errors. Accordingly, such an algorithm, considering the above two cases together, that is, an algorithm applying temporal error concealment, as well as spatial error concealment, is proposed.
In detail, temporal error concealment is performed, and a macroblock having the minimum value of the sum of absolute differences between boundary values is selected. If the sum of absolute differences is greater than a predetermined reference value, an extracted macroblock value is discarded, and conventional spatial error concealment, currently used in H.264, is used, whereas, if the sum of absolute differences is equal to or less than the reference value, a lost macroblock value is replaced with a macroblock value obtained through temporal error concealment.
Of the above algorithms, the algorithms proposed by Jung et al. and Lee et al. execute only a temporal error concealment algorithm. Further, the algorithms proposed by Xu et al. and Panos et al. adopt a hybrid method, but select only a single method from among temporal error concealment and spatial error concealment, and apply the selected method.
Therefore, since the above-described conventional methods realize only the exclusive application of any one of temporal error concealment and spatial error concealment, they are limited in that it is difficult to provide a service having a required quality in a network base in which a required specification is high, but a provided service environment is relatively poor, as in the case where real-time video is intended to be transmitted in a network environment in which a packet loss rate is high, and a bit rate is relatively low.